Advances on Weightless Neural Systems

Random Access Memory (RAM) nodes can play the role of artificial neurons that are addressed by Boolean inputs and produce Boolean outputs. The weightless neural network (WNN) approach has an implicit inspiration in the decoding process observed in the dendritic trees of biological neurons. An overview on recent advances in weightless neural systems is presented here. Theoretical aspects, such as the VC dimension of WNNs, architectural extensions, such as the Bleaching mechanism, and novel quantum WNN models, are discussed. A set of recent successful applications and cognitive explorations are also summarized here. 1 From n-tuples to artificial consciousness It has been 55 years since Bledsoe and Browning [20] introduced the n-tuple classifier, a binary digital pattern recognition mechanism. The pioneering work of Aleksander on RAM-based artificial neurons [1] carried forward Bledsoe and Browning’s work and paved the path of weightless neural networks: from the introduction of WiSARD (Wilkes, Stonham and Aleksander Recognition Device) [2][3], the first artificial neural network machine to be patented and commercially produced, into the 90’s, when new probabilistic models and architectures, such as PLNs, GRAMs and GNUs, were introduced and explored [4][5][6][7][8][9]. A natural drift into cognitive and conscious architectures followed, due to the work of Aleksander and colleagues [10][11][12][13][14][15]. A brief but more detailed history of weightless neural systems can be found in [16]. While Braga proposed a geometrical and statistical framework to model the state space of pattern distribution in the n-dimensional Boolean space [22], Bradshaw introduced the use of statistical learning theory tools in the analysis of the n-tuple classifier and related weightless neural models [23]. Interestingly, Bradshaw found out that the VC dimension of the n-tuple classifier suggests much poorer generalization capabilities than found in practice, which also motivated the production of the effective VC dimension for this weightless model [24][25]. The high VC dimension of the n-tuple classifier looks underexplored since saturation of RAM nodes contents often happens if a relatively small n value is chosen, i.e., when the size of the training set is large enough to allow for writing 1’s in most of the RAM nodes positions. The introduction of the bleaching mechanism [40][56] was possible via extending RAM nodes from one-bit positions to counters able to register the number of times a particular RAM position was accessed during the training phase. This extension follows early probabilistic weightless models, such as PLNs and GRAMs, where RAM positions can also hold values different from 0’s and 1’s that are interpreted as firing probabilities. In the next section, bleaching is explained together 497 ESANN 2014 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 23-25 April 2014, i6doc.com publ., ISBN 978-287419095-7. Available from http://www.i6doc.com/fr/livre/?GCOI=28001100432440.